Practice Midterm 2

QTM 220

Problem 1

Suppose we have a sample $Y_1 \ldots Y_n$ drawn with replacement from a population $y_1 \ldots y_m$ with mean $\mu = \frac{1}{m}\sum_{j=1}^m y_j$. We have it in an vector $Y$ of length $n$. We run this code.

B=1000
Y.bar = rep(NA,B)
for(b in 1:B) {
  Y.bar[b] = mean(sample(Y,size=n,replace=TRUE))
}

Part A

What does this code calculate? State it in words and mark it with an ‘a’, and an arrow or bracket if necessary for clarity, on the appropriate plot above.

mean(Y) - 1.96* sd(Y)/sqrt(n)
mean(Y) + 1.96* sd(Y)/sqrt(n)

Part B

How could you calculate roughly the same thing as in Part A using sd(Y.bar) and not sd(Y)?

Problem 2

Suppose $Y_1 \ldots Y_n$ are sampled with replacement from a population with mean $\mu=1$ and variance $\sigma^2=1$. In Midterm 1, we discussed the estimator $\tilde{Y}_k = \frac{1}{n-k} \sum_{i=1}^n Y_i$ for $k=2$. Here, we’ll talk about that case and the case $k=\sqrt{n}$.

Part A

For both estimators $\tilde{Y}_2$ and $\tilde{Y}_{\sqrt{n}}$, for each of the three sample sizes $n=10$, $n=10^2=100$, and $n=10^4=10,000$, do the following.

Calculate the estimator’s mean $\E[\tilde{Y}_k]$ and variance $\Var[\tilde{Y}_k]$.
Sketch the estimator’s sampling distribution. Label your axes clearly and include $\mu=1$ in your sketch.
On top of your sketches, illustrate how you’ve calculate the coverage probability of the interval estimate $\tilde{Y}_k \pm 1.96 \sqrt{\Var[\tilde{Y}_{k}]}$.
Give a rough approximation of this coverage probability. You don’t have to do this well—just do it by eye. Is it about 95%? Is it above 50%? That’s the kind of precision I’m looking for.

You should have 6 sketches with corresponding means, standard errors, and coverage probabilities. That’s more than we’d ask for on the real exam. If you get the idea and want to save time by skipping the $n=10^2$ case, that’s fine.

Problem 3

In the plot below, I’m showing some data from a (fake) survey of people in your city who’ve recently finished with school and begun working. For whatever reason, nobody who attends school in this city goes to college, so everyone finishes with between 8 and 12 years of education. There are 10,000 of these people, and we mailed out (and received back) 2000 surveys, sampling from this population of 10,000 with replacement. The x-axis shows the survey recipient’s years of education, the y-axis sjpws their income, the points with arms are within-column means and standard deviations, and the heights of the bars are proportional to the proportion of recipients with each level of education.

Part A

Consider two summaries of the incremental value of a year of high school in this sample.

\[ \begin{aligned} \hat\theta_{\text{years}} &= \frac{\sum_{x \in 9 \ldots 12} \qty{\hat\mu(x) - \hat\mu(x-1)}}{4} \\ \hat\theta_{\text{people}} &= \frac{\sum_{i:X_i \in 9 \ldots 12} \qty{\hat\mu(X_i) - \hat\mu(X_i-1)}}{\sum_{i:X_i \in 9\ldots 12} 1} \\ \end{aligned} \]

Which is larger? Explain how you know.

These survey recipients went to two schools: School 0 (circles) and School 1 (triangles).¹

Children in your district go to School 0. Your cousin, who lives in a district that goes to School 1, encourages you to pretend that you and your family live with them so that your child can go to School 1. This is a thing that happens, at least on the TV show ‘Friday Night Lights.’ Their argument is that School B must be better because the average income of students who have attended it is clearly higher. Check out this confidence interval for the difference in mean incomes of students who have attended the two schools, they say.

Delta.hat = mean(Y[W==1]) - mean(Y[W==0])

Delta.hat.star = replicate(1000, {
  J.star = sample(1:n, size=n, replace=TRUE)
  W.star = W[J.star]
  X.star = X[J.star]
  Y.star = Y[J.star]
  
  mean(Y.star[W.star==1]) - mean(Y.star[W.star==0])
})

interval.width = width(Delta.hat.star)
confidence.interval = c(Delta.hat - interval.width/2, 
                        Delta.hat + interval.width/2)
confidence.interval

[1]  7323 12428

They even based their calculation on one of your QTM 220 homework solutions, so it’s got to be legit. School 1 is at least $7323 better.

You, on the other hand, look at the data and claim that a year at School 0 is worth just as much as a year at School 1. And you’re right. You calculate school-specific versions of $\hat\theta_{\text{years}}$, bootstrap the difference between them, and find convincing evidence that the incremental value of a year at each school is the roughly the same.

your.summary = function(W,X,Y) {
  muhat = Vectorize(function(w,x) { mean(Y[W==w & X==x]) })
  (mean(muhat(1,9:12)-muhat(1,8:11))) - (mean(muhat(0,9:12)-muhat(0,8:11)))
}

theta.hat.years.difference = your.summary(W,X,Y)

theta.hat.years.difference.star = replicate(100, {
  J.star = sample(1:n, size=n, replace=TRUE)
  your.summary(W[J.star], X[J.star], Y[J.star])
})
 

interval.width = width(theta.hat.years.difference.star)
confidence.interval = c(theta.hat.years.difference - interval.width/2, 
                        theta.hat.years.difference + interval.width/2)
confidence.interval

[1] -1518  1302

Part B

Explain, in terms your cousin can understand, how it’s possible that the average income of students who have attended School 1 is higher than the average income of students who have attended School 0, but the average incremental value of a year at both schools is the same.

Problem 4

Below, I’ve shown least squares fits of six regression models to data from a randomized experiment. In this experiment, patients in our population were assigned to treatment ($w=1$, green) or control ($w=0$, red) with probability depending only on whether their age ($x$) was less than or equal to 65. Then, to get the patients who actually participated in our experiment, we sampled with replacement from this population.

Here are the models. \[ \begin{aligned} \model_1 &= \qty{ m(w,x) = a + bx \ : \ \text{ any numbers }\ a,b } && \text{ one line } \\ \model_2 &= \qty{ m(w,x) = a(w) : \ \text{ any function }\ a(w) } && \text{ horizontal lines } \\ \model_3 &= \qty{ m(w,x) = a(w) + bx \ : \ \text{ any function }\ a(w) \ \text{ and number }\ b } && \text{ parallel lines } \\ \model_4 &= \qty{ m(w,x) = a(w) + b(w)x \ : \ \text{ any functions }\ a(w),\ b(w) } && \text{ lines } \\ \model_5 &= \qty{ m(w,x) = a(w) + b \operatorname*{1_{\le 65}}(x) \ : \ \text{ any function } a(w) \ \text{ and number }\ b } && \text{ additive piecewise-constant } \\ \model_6 &= \qty{ m(w,x) = a(w) + b(w) \operatorname*{1_{\le 65}}(x) \ : \ \text{ any functions } a(w),\ b(w) } && \text{ piecewise-constant } \end{aligned} \] for \[ 1_{\le 65}(x) = \begin{cases} 1 & \text{ if } x \le 65 \\ 0 & \text{ otherwise} \end{cases}. \]

Part A

For each plot, identify the model that’s plotted. That is, for each plot A…F, give a model number 1…6.

Suppose we want to estimate the average treatment effect on the treated, \[ \text{ATT} = \frac{1}{m_1}\sum_{j:w_j=1} \qty{ \mu(1,x_j) - \mu(0,x_j)} \qfor \mu(w,x) = \frac{\sum_{j: w_j,x_j=w,x } y_j}{\sum_{j: w_j,x_j=w} 1} \qand m_w = \sum_{j: w_j=1} 1. \]

You do this by using a sample version of the same summary and using our least squares predicition $\hat\mu(w,x)$ in place of $\mu(w,x)$. \[ \widehat{\text{ATT}} = \frac{1}{N_1}\sum_{i:W_i=1} \qty{ \hat\mu(1,X_i) - \hat\mu(0,X_i) } \qfor N_w = \sum_{i: W_i=1} 1. \]

Part B

For each model, state whether this’ll give you an unbiased estimate of the ATT. If your answer is yes, prove it. If you answer is no, explain why not.

Important

This is Extra Credit. We’re not going to review this one in class. However, each correct answer is worth 1 point on your exam, so you can get up to 6 points of extra credit for this.If you get the ‘whether’ part right but the ‘why’ part wrong, you’ll get 1/2 point. I’ll provide you with the data and code used to generate the plot above, so you should be able to use to check your claims.

Tip

Recall from Homework 4 that the least squares residuals are orthogonal to all functions in the model, i.e., \[ \sum_{i=1}^n \qty{Y_i - \hat\mu(W_i,X_i)} m(W_i,X_i) = 0 \qqtext{ for every } m \in \model. \]